This invention relates to a computer-executed procedure for searching an image database that includes video by use of queries based on the image characteristics of segmented video frames.
In the prior art, the storage, indexing, and retrieval of video images in a computer database is known. See, for example, the article by M. Flickner, et al entitled Query by Image and Video Content: The QBIC System, in COMPUTER, September 1995, pp. 23-32. This article describes an image database that includes still images and video. For query processing of video in a database, images that represent contents of stored video are used. These images are typically in the form of frames, and may comprise individual frames extracted from a sequence of frames (key frames), as well as synthesized frames that are composites of such sequences. Such synthesized frames are very desirable in searching video since they include data respecting motion in the sequences of frames from which they are composed. This permits queries based upon motion, among other things.
Video frames (either key frames or synthesized frames) may be processed by being segmented into objects. In the prior art, an object is a dominant figure in an image. Objects may be identified (automatically or manually) as members of a restricted class of predetermined images, or by user-guided algorithms. In the described prior art image database, queries are allowed on objects, frames, or any combination thereof. In this regard, a query is presented in a form called a "query image" having a semantic significance ("Find images with a yellow star"). The query specifies values of image characteristics such as color, texture, and shape. Similarity is supported by specification of location, distance, and weight. A query response is returned that includes a user-defined number of images that most closely match the query, ranked in order of similarity, based upon the specified image characteristic values and the similarity factors.
Unfortunately, prior art video image databases provide a query capability that is limited to segmentation of images and frames into objects alone. Manifestly, a richer segmentation vocabulary would support greater precision in query specification.